AD-A011  929 


IMAGE  ANALYSIS  AND  MODELING 
Paul  A.  Wintz,  et  al 
Purdue  University 


Prepared  for: 

Rome  Air  Development  Center 

Defense  Advanced  Research  Projects  Agency 


DISTRIBUTED  BY: 


National  Technical  Information  Service 
U.  S.  DEPARTMENT  OF  COMMERCE 


AD  AO  11  9 


198032 

l 


RADC-TR-75-144 
Technical  Report 
03  ' Hay  1975 

w 

IMAGE  ANALYSIS  AND  MODELING 

Purdue  University 


Sponsored  By 

Defense  Advanced  Research  Projects  Agency 
ARPA  Order  No.  2893 


Approved  for  public  release; 
distribution  unlimited. 


J3  0 q 

mnsimm 

Ijj^JUL  7 1975 

A 


The  views  and  conclusions  contained  in  this  document  are  those  of  the 
authors  and  should  not  be  interpreted  as  necessarily  representing  the 
official  policies,  either  expressed  or  implied,  of  the  Defense 
Advanced  Research  Projects  Agency  or  the  U.  S.  Government. 


Rome  Air  Development  Center 
Air  Force  Systems  Command 
Griff iss  Air  Force  Base,  New  York  13441 

Reproduced  by 

NATIONAL  TECHNICAL 
INFORMATION  SERVICE 

U 5 Deportment  of  Commerce 
Springfield  VA  22151 


SECURITY  CLASSIFICATION  of  This  PAGE  Cl*?>en  Data  Entered) 


REPORT  DOCUMENTATION  PAGE 


4.  T|  TLE  (and  Subtitle) 


IMAGE  ANALYSIS  AND  MODELING 


7.  AUTHOR'*; 


Paul  A.  Wintz 
T.  S.  Huang 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


3 RECIPIENT’S  catalog  number 


S.  TYPE  OF  REPORT  & PERIOD  COVEREO 

Quarterly  Progress  Report 
1 Nov  74  - 31  Jan  75 


6 PERFORMING  ORG.  REPORT  NUMBER 

TR-EE  75-3 


6 CONTRACT  OR  GRANT  NUMBERf*) 

F30602-75-C-0150 


9 PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

Purdue  University 

School  of  Electrical  Engineering 

West  Lafayette  IN  47907 


I I CONTROLLING  OFFICE  NAME  AND  ADDRESS 

Defense  Advanced  Research  Projects  Agency 
1400  Wilson  Blvd 
Arlington  VA  22209 


M MONITORING  AGENCY  NAME  8 ADDRESSfU  dtttercnt  trow  Controlling  O liter) 

Rome  Air  Development  Center  (IRRO) 

Griffiss  AFB  NY  13441 


te.  distribution  statement  foi  thi,  Report) 

Approved  for  public  release;  distribution  unlimited. 


10.  PROGRAM  ELEMENT.  PROJECT,  TASK 
AREA  8 WORK  UNIT  NUMBERS 

61101E 

28930001 


12.  REPORT  DATE 

May  1975 


13  NUMBER  OF  PAGES 


IS.  SECURITY  CLASS,  (ot  this  report) 


UNCLASSIFIED 


ISa.  DECLASSIFICATION  DOWNGRADING 
SCHEDULE 

N/A 


17.  DISTRIBUTION  STATEMENT  foi  the  obstruct  entered  In  Block  20,  II  dillerent  from  Report) 

Same 


ie.  supplementary  notes 


RADC  Project  Engineer: 

John  M.  Trossbach,  Capt,  USAF 
AC  315  330-2353 


19.  KEY  WORDS  (Continue  on  reverse  side  II  neceeemry  end  Identity  by  block  number) 


Image  Processing 
Digital  Image  Processing 
Image  Restoration 
Image  Coding 


Pattern  Recognition 
Syntactic  Pattern  Recognition 
Image  Segmentation 


20.  ABSTRACT  (Continue  on  reverae  aids  II  naceaaary  and  Identify  by  block  number) 

This  report  summarizes  the  results  of  the  research  program  on  Image 
Analysis  and  Modeling  supported  to  the  Advanced  Research  Projects  Agency  under 
Rome  Air  Development  Center  Contract  F30602-75-C-0150  for  the  three  month 
period  1 November  1974  to  31  January  1975.  The  objective  is  to  achieve  a 
better  understanding  of  image  structure  and  to  use  this  knowledge  to  develop 
improved  image  models  for  use  in  image  analysis  and  processing  tasks  such  as 
information  extraction,  image  enhancement  and  restoration,  coding,  etc.  The 

(Cont'd) 


DD  | JAN *73  1473  EDITION  OF  I NOV  65  IS  OBSOLETE 


liOglittiSI 


\ SECURITY  CLASSIFICATION  OF  THIS  PAGE  fRTien  Dele  Entered) 


SECURITY  CLASSIFICATION  OF  THIS  PAGEfHTi^i  Polo  Entered) 

ultimate  objective  of  this  research  is  to  form  a basis  for  the  development 
of  technology  relative  to  military  applications  of  machine  extraction  of 
information  from  aircraft  and  satellite  imagery  of  the  earth's  surface.  Work 
reported  here  includes:  image  segmentation  and  decomposition,  edge  detection 

texture  analysis,  clustering,  feature  extraction,  coding  and  restoration. 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGEOFften  Dora  Entered) 


This  -eport  has  bacn  levieved  by  the  RADC  Inforiation  Ofiice  (01)  and 
is  releasable  to  the  National  Technical  Information  Service  (NTIS).  At  OTIS 
It  will  be  releasable  to  the  general  public.  Including  foreign  nations. 

This  report  has  been  reviewed  and  at\>roved  for  publication. 


APPROVED 


JOHN  M.  TROSSBACH,  JR.,  CAPT,  USAF 
Project  Engineer 


Do  not  return  this  copy.  Retain  or  destroy 


IMAGE  ANALYSIS  AND  MODELING 


Paul  A.  Wintz 
T.  S.  Huang 


Contractor:  Purdue  University 

Contract  Number:  F30602-75-C-0150 

Effective  Date  of  Contract:  1 November  1974 

Contract  Expiration  Date:  31  October  1975 

Amount  of  Contract:  $427,020.00 

Program  Code  Number:  5D30 

Period  of  Work  Covered:  1 Nov  74  - 31  Jan  75 


Principal  Investigators: 
Phone : 


Dr.  Paul  A.  Wintz  and 
Dr.  Thomas  S.  Huang 
317  749-2540 


Project  Engineer: 

Phone : 


Capt  John  M.  Trossbach 
315  330-2353 


Approved  for  public  release; 
distribution  unlimited. 


This  research  was  supported  by  the  Defense 
Advanced  Research  Projects  Agency  of  the 
Department  of  Defense  and  was  monitored  by 
Capt  John  M.  Trossbach,  RADC  (IRRO)  Griffiss 
AFB  NY  13441. 


TABLE  OF  CONTENTS 


Page 

1.  RESEARCH  SUMMARY  AND  OVERVIEW  1 

II.  RESEARCH  PROJECTS 

Image  Segmentation 2 

by  P.  A.  WIntz,  P.  H.  Chen  and  P.  Miller 

Image  Modeling  by  l ecompos 1 1 1 on 3 

by  T.  S.  Huang  and  J.  Burnett 

Syntactic  Pattern  Recognition  6 

by  W.  J.  Kelly 

Picture  Statistics  and  Textures  11 

by  C.  Myers  and  0.  R.  Mitchell 

Image  Restoration  by  Recursive  Estimation  15 

by  A.  C.  Kak  and  D.  Panda 

Image  Modeling  by  Independent- Increment  Processes,  II  . . . 18 

T.  S.  Huang  and  J.  Lban 

Satellite  Image  Formation  Model  25 

by  0.  R.  Mitchell  and  P.  L.  Chen 

Iterative  Image  Restoration,  II  27 

by  T.  S,  Huang  and  S.  P.  Berger 

A Branch  and  Bound  Clustering  Algorithm  36 

by  K.  Fukunaga  and  P.  M.  Narendra 

Application  of  Singular  Value  Decomposition  to 

Pattern  Recognition 1,] 

by  K.  Fukunaga  ar.d  G.  V,  Sherman 

III.  LABORATORY  FACILITIES 1,6 

IV.  SOFTWARE  DEVELOPMENT  FOR  IMAGE  PROCESSING I»7 

V.  -PERSONNEL 1,8 

VI.  PUBLICATIONS 1,9 

VII.  RELATED  ACTIVITIES 50 


iii 


IMAGE  ANALYSIS  AND  MODELING 
RESEARCH  SUMMARY  AND  OVERVIEW 

In  this  report  we  summarize  our  research  activities  during  the  three 
month  period  November  I,  197A  to  January  31,  1975.  Our  research  program 
r ted  October  1,  1973.  During  the  first  year  we  concentrated  our  efforts 
on  basic  research  in  image  structure  analysis  and  image  modeling.  During 
the  second  year  we  shall  continue  our  basic  research,  but  in  the  meantime 
also  look  into  several  applications  In  the  information  extraction  area.  We 
hope  that  on  the  one  hand  the  results  of  our  basic  research  could  find 
practical  applications,  and  on  the  other  hand  problems  arise  from  practical 
situations  would  guide  the  direction  of  our  basic  research. 

In  has  research,  we  will  emphasize  Image  segmentation  and  decomposi- 
tion, edge  detection,  texture  analysis,  linguistic  and  syntactic  approach, 
and  feature  extraction.  The  applications  we  shall  look  into  Include: 
information  extraction  for  the  purpose  of  tracking  flying  objects,  matching  of 
similar  but  not  identical  Images,  and  detection  of  airports  In  ERTS  pictures. 

As  part  of  our  research  program  we  are  putting  together  a computer 
system  which  will  be  a node  on  the  ARPA  computer  network.  The  system  is 
working  locally  and  will  be  on  the  ARPANET  in  the  near  future.  The  avail- 
ability of  such  a system  will  affect  our  research  In  two  respects.  First, 
because  the  system  includes  a flexible  digital  display  unit,  we  can  very  con- 
veniently study  man-machine  interactive  techniques.  Second,  because  of  the 
existence  of  l«-.rge  computers  (such  as  the  IBM  360/91,  the  CDC  7600,  and  the 
ILLIAC  A)  on  the  ARPANET,  we  can  afford  to  study  algorithms  which  may  require 
a large  amount  of  computation. 


IMAGE  SEGMENTATION 

P.  A.  Wintz,  P.  H.  Chen,  and  P.  Miller 


As  reported  earlier  [I),  we  have  developed  a computer  algorithm  (BLOB) 
to  segment  an  Image  in*o  regions.  This  algorithm  has  been  proven  very 
successful  In  some  classification  and  coding  applications.  We  are  currently 

working  on  extensions  of  this  BLOB  algorithm  to  make  It  useful  for  other 
problems.  In  particular: 

(I)  We  are  developing  an  algorithm  for  segmenting  mul tlspectral  images 
into  multidimensional  ( 3 or  higher  dimensional)  blobs. 

(")  The  or'9inal  BL0B  algorithm  uses  the  me*n  and  variance  of  gray 
levels  to  group  points  into  regions.  We  are  looking  Into  the  use  of  other 
statistics  which  may  be  more  suitable  for  other  applications. 


[1] 
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IMAGE  MODELING  BY  DECOMPOSITION 
T.  S.  Huang  and  J.  Burnett 

I . STATEMENT  OF  THE  PROBLEM 

We  -3 re  developing  improved  image  models  based  on  decomposition  which 
could  lead  to  break  throughs  In  image  processing. 

Although  there  are  many  examples  of  success  in  image  processing,  a large 
number  of  problems  remain  unsolved  or  only  partially  solved.  Consider 
specifically  the  case  of  image  restoration  where  one  tries  to  compensate  for 
the  degradations  suffered  by  the  image  due  to  camera  motion,  lens  aberrations, 
atmospheric  turbulence,  etc.,  and  the  ever-present  detector  noise.  Past 
techniques  In  Image  restoration  [l]  succeeded  In  reducing  image  noise  but 
generally  failed  in  sharpening  up  the  blurred  edges  of  objects  in  the  image. 
The  main  reason  for  this  failure  is  that  in  most  of  these  techniques  the  image 
is  modeled  by  a stationary  second-order  stochastic  process  with  an  exponential 
autocorrelation  function  [2,3].  It  can  be  shown  [4]  that  sample  functions 
of  such  stochastic  processes  are  necessarily  continuous  - they  cannot  contain 
sudden  jumps.,  it  is  therefore  not  suprising  that  restoration  techniques  based 
on  this  kino  of  image  model  yield  fuzzy  pictures. 

We  believe  that  the  key  to  getting  a good  restoration  lies  in  the  de- 
velopment of  an  improved  image  model.  Specifically,  an  image  should  be  de- 
composed into  several  parts.  Then  each  part  is  treated  separately  according 
to  Its  own  characteristics.  As  a first  attempt,  we  will  consider  an  image  as 
the  sum  of  a pure-jump  process  (representing  the  edges  in  the  Image)  and  a 
piecewise  Markov  process.  The  jumps  will  divide  the  Image  into  regions. 

Each  of  these  regions  Is  modeled  by  a Markov  process  with  its  own  parameters. 
For  restoration,  Wiener  or  Kalman  filters  could  be  used  for  the  Markovian 
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regions,  while  new  techniques  perhaps  hwe  to  be  developed  for  the  pure- jump 
process  [5,6]. 

I I . SIGNIFICANCE  OF  THE  PROBLEM 

The  development  of  improved  image  models  based  on  decomposition  will  not 
only  effect  significant  progress  in  image  restoration  but  also  have  a pro- 
found influence  on  all  areas  of  image  processing. 

For  example,  in  efficient  coding  if  we  should  succeed  in  decomposing  the 
image  into  edges  and  background,  the  two  parts  could  be  coded  differently  - 
perhaps  chain  coding  [7]  for  the  edges  and  transform  coding  [8]  for  the  back- 
ground. In  pattern  recognition,  suitable  decomposition  of  an  image  could  be 
a crucial  first  step  In  feature  extraction.  And  in  computer  graphics,  image 
decomposition  seems  a good  approach  to  the  generation  and  display  of  con- 
tinuous-tone pictures-thus  the  'block  world"  of  the  artificial  intelligence 
community  could  be  displayed  as  edges  plus  shading.  Finally,  the  search  for 
objective  (mathematical)  measures  of  image  quality  may  be  made  easier  if 
different  criteria  are  used  for  the  different  components  of  an  image. 

All  in  all,  we  believe  that  study  on  image  modeling  basec  on  decomposi- 
tion could  lead  to  important  break  throughs  in  Image  processing. 

III.  PLAN  OF  RESEARCH 

The  aim  of  our  research  Is  to  develop  Improved  Image  models  based  on 
decomposition  and  to  apply  these  models  to  image  processing.  We  are  proceed- 
ing along  the  following  overlapping  steps: 

1)  Develop  mathematical  theory  for  Image  decomposition.  We  are  particularly 
interested  in  decomposing  an  Image  Into  a pure-jump  process  plus  a piecewise 
Markov  process.  The  pieces  of  the  latter  may  further  be  classified  Into 
"slowly-changing  background"  and  "textures". 

2)  Develop  techniques  of  estimating  model  parameters  from  a given  image  which 

k 


could  be  blurred  and  noisy. 

3)  Develop  restoration  techniques  for  the  various  components  of  a decomposed 
image.  Special  attention  will  be  paid  to  the  particular  characteristics  of 
each  component. 

*0  Apply  the  image  model  and  the  restoration  techniques  to  real-life  degraded 
images. 

5)  Apply  the  Image  model  to  coding,  classification,  and  information  extraction. 

REFERENCES 
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SYNTACTIC  PATTERN  RECOGNITION 
W.  J.  Kelly 
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I.  INTRODUCTION 

During  the  past  several  years  there  has  been  increasing  interest  in 
Syntactic  Picture  Recognition  or  Description.  Calling  upon  the  theory  of 
Formal  Languages,  this  approach  aims  at  producing  picture  descriptions  which 
exploit  structural  relationships  in  the  picture.  Both  theoretical  development 
and  application  have  been  reported  in  the  literature.  A short  review  and 
references  may  be  found  in  section  II  of  this  surmiary. 

The  immediate  goal  of  this  research  is  to  Identify  concepts  and  tech- 
niques to  solve  military  information  extraction  problems.  Initial  work  has 
concentrated  on  the  use  of  such  techniques  for  edge  detection  and  the 
Identification  of  airports  in  ERTS  pictures.  In  particular,  the  tools  in- 
c*ude'  stochastic  grammars  to  handle  Image  smearing  and  noise,  and,  ii) 
web_ grammars  to  exploit  fully  the  structural  relationships  In  the  picture. 

1 1 • SYNTACTIC  PATTERN  RECOGNITION  AND  EDGE  DETECTION 

The  application  of  language  theory  to  pattern  recognition  goes  back  to 
the  early  sixties,  when  It  was  recognized  that  many  applications  Involve 
picture  "structure"  that  was  not  expressible  with  convention!  techniques  [-4]. 
The  extensive  mathematical  formalism  of  linguistic  theory  appears  to  be  a 
natural  way  of  developing  this  structure.  Two  approaches  are  possible,  de- 
pending on  the  kind  of  "primitives"  chosen  to  correspond  to  the  "terminals" 
of  language  theory.  The  first  approach  defines  the  primitives  qui te  general  i y 
while  the  second  uses  elements  of  a grid  as  primitives  with  simple  adjacency 
reiationsh  ps  [2]. 
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The  second  approach  allows  relatively  simple  adaptation  of  conventional  (string- 
generating) language  theory,  but  the  representations  may  be  ambiguous  and  are 
seld  n "transparent".  The  first  approach  allows  sophisticated  relationships 
to  he  described,  but  the  theory  must  be  established  independently,  and  the 
problem  of  detecting  complex  relationships  in  a picture  is  non-trlvial. 

Use  of  these  descriptions  to  identify  patterns  - "parsing"  In  linguistics  - 
presents  a major  difficulty.  Formal  linguistic  parsing,  e.g.  a FORTRAN 
compiler,  makes  little  allowance  for  small  "deviations"  in  a string,  and  such 
deviations  will  cause  rejection.  Any  useful  picture  wMl  contain  some  noise 
and  "smearing",  and  this  may  cause  the  parsing  to  fail  even  though  the  structure 
is  "evident"  to  a human  observer.  The  problem  may  be  avoided  to  some  extent 
by  extensive  pre-processing  of  the  data,  but  this  is  expensive,  inefficient 
and  may  destroy  significant  Information.  A promising  approach  is  to  asso' ! • 
ate  probabilities  with  the  grammar  and  language,  forming  sc-ca lied  "stochastic" 
grammars  and  languages  which  will  allow  the  successful  parsing  of  patterns 
which  deviate  "slightly"  from  the  ideal.  Considerable  work  on  stochastic 
picture  grammars  has  been  done  at  Purdue  [3,10]. 

The  most  promising  syntactic  pattern  formal Isn  using  "general"  primi- 
tives is  the  "web  grammar"  approach  [7].  There  are  several  formulations, 
but  the  idea  is  to  describe  graph  structure  where  the  nodes  represent  "features" 
of  the  picture  and  the  edges  represent  "relations".  The  "rewriting  rules"  re- 
place "subgraphs"  with  more  complicated  subgraphs.  There  have  beer  many  re- 
sults reported  which  extend  string-grammar  theorems  to  web  grammar,  but  much 
remains  to  be  done  [],4,6,8]. 

One  application  where  these  techniques  might  be  profitably  employed  Is  In 
the  area  of  "edge  detection"  [9].  The  general  problem  is  to  find  that  which 
human  observers  would  call  an  "edge,  and  often  the  "structure"  in  the  picture 
could  be  usefully  employed.  The  principal  difficulties  are  on  the  one  hand, 


"smearing"  of  the  edges  and  r.oise,  and  on  the  other,  detection  of  boundaries 
between  areas  characterized  more  by  "texture"  differences  than  by  grey  level 
di fferences. 

Ml.  TENTATIVE  RESEARCH  PLAN 

One  couid  continue  extending  "picture  grammar"  theory  along  the  lines  cf 
phrase-structure  grammars.  However,  it  is  not  evident  that  the  extens ions  wi  1 i 
be  applicable  to  the  problems  of  pattern  recognition.  Other  approaches,  such 
as  "semantic"  and  "transformational  grammar"  pattern  recognition,  have  been 
proposed,  but  little  solid  theory  exists  so  it  is  difficult  to  judge  their 
utility.  To  gain  insight  into  the  applicability  of  the  present  theory  to  a 
real  problem,  the  "edge  detection"  area  will  be  investigated.  These  problems 
exhibit  both  structure  and  uncertainty  and  thus  seem  amenable  to  the  tools  of 
web  grammars  and  stochastic  grammars.  Two  aspects  may  be  distinguished. 

Insofar  as  the  edges  are  part  of  a larger  "structured"  pattern,  relationships 
more  subtle  than  adjancency  may  prevail  and  the  formalism  of  web  grammars  seems 
appropriate.  However,  the  phenomena  of  noise  and  "smearing"  are  often  local 
and  stochastic  "global  structure"  grammars  would  not  seem  useful  in  handling 
the  problem.  Moreover,  the  physical  location  of  the  edges  in  the  picture  may 
be  significant  and  then  the  adjacency  relation  would  be  important.  One  should 
also  consider  the  processing  problem.  For  maximum  efficiency,  it  is  probable 
that  parallel  processing  will  be  employed  in  application  involving  pattern 
structure;  parts  of  a pattern  may  be  intelligible  only  in  light  of  other  parts. 
We  propose  to  exploit  this  parallel  processing  capability  by  employing  a "two- 
level"  scheme  to  detect  edges  in  noise  where  there  is  a priori  knowledge  of 
ome  structure  in  the  pattern.  The  basic  idea  is  to  separate  the  problem  of 
'.tecting  the  edges  in  noise  from  that  of  detecting  the  edge  structure,  but 
Improve  the  efficiency  of  each  solution  by  transferring  information  from 
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one  to  another.  To  locate  noisy  or  "smeared"  edges,  a parser  for  a stochastic 
string  generating  grammar  will  be  us*d,  but  Its  operation  will  be  controlled 
by  a web  grammar  parser  that  recognizes  the  edge  structure.  This  assumes 
a parallel  processing  capability  so  that  the  two  levels  may  proceed  simulta- 
neously and  guide  each  other.  The  use  of  stochastic  web  grammars  would  not 
be  intende j to  handle  "local"  noise  and  smearing,  but  rather  any  such  phe- 
nomena which  have  a 'fuctural  character  (e.g.  edges  obscured  by  shadows). 

Presently  Purdue  does  not  have  a true  parallel  processing  capability, 
but  the  long-range  usefulness  of  the  method  nukes  It  attractive  to  employ 
either  a pa ral lei -processing  simulator  such  as  PAX  [ 1 1 ] or  the  1LL1AC  IV 
through  the  facilities  of  the  ARPANET. 

These  concepts  will  be  developed  using  data  from  hlghtly-structured 
pictures.  In  o to  avoid  Irrelevant  detail.  Pictures  of  straight-edged 
3-D  blocks  have  been  employed  for  previous  edge-detection  research,  thus 
giving  a "benchmark"  and  the  initial  syntactic  research  will  concentrate  on 

these. 

IV.  IMMEDIATE  GOALS 

1)  Development  of  a web  grammar  for  a class  of  "block"  pictures.  High 
priority  should  be  given  to  ease  of  parsing:  l.e.  minimizing  "backtracking". 

.11)  Development  of  a stochastic  string  grammar  to  represent  smeared  or 
noisy  edges  of  block  pictures.  This  may  be  an  extension  of  Swain  and  Fu's 
work  [10]. 

Ill)  Consideration  of  the  type  and  method  of  Information  transfer 

] 

between  the  two  "levels".  Presumably  the  web  grammar  would  modify  the  prob- 
abilities of  the  stochastic  grammar,  and  each  parallel  execution  of  the 
string-grammar  passe  would  Inform  the  web  grammar  passes  of  al  ready  "1  dent  1 fled" 

edges. 
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iv)  Consideration  of  an  "Initialization"  scheme,  since  the  parsers 
must  have  something  presented  to  them.  This  Is  related  to  the  choice  of  an 
edge  segment  Identification  scheme.  Presumably  standard  statistical 
techniques  will  be  employed.  The  "upper-level"  parse  would  not  merely 
guide  the  "lower-level"  parse  but  may  also  search  for  edges  not  detected 
by  ini tlal I zation. 


REFERENCES 

[1]  J.  M.  Bray».r  and  K.  S.  Fu,  "Some  Properties  of  Web  Grammars,"  TR-EE- 
74--19,  Purdue  University,  1974. 

[2]  K.  S.  Fu  and  P.  H.  Swain,  "On  Syntactic  Pattern  Recognition,  Software 
Engineering,  Vol.  2,  (ed.  Tou),  Academic  Press,  New  York,  1971. 

[3]  T.  Huang  and  K.  S.  Fu,  "Stochastic  Syntactic  Analysis  and  Syntactic 
Pattern  Recognition,"  TR-EE-72-5,  Purdue  University,  1972. 

[4]  U.  G.  Montanari,  Separable  graphs,  planar  graphs,  and  web  grammars, 

Inf.  and  Cont.  Vol.  16,  pp.  243-67,  1970. 

[5]  R.  Marislmhan,  A linguistic  approach  to  pattern  recognition,  Dlq.  Comp. 
Lab  Report  #121,  Univ.  of  Illinois,  Urbana,  1 962 . 

[6]  J.  L.  Pfaltz,  Web  grammar  and  picture  description.  Comp.  Graph,  and 
Image  Proc.  Vol.  1,  pp.  193-220,  1972. 

[7]  J.  L.  Pfaltz  and  A.  Rosenfeld,  Web  Grammars,  Pros.  1st  Int.  Joint, 

Conf.  Art.  Instell,  Wash.  D.C.,  pp.  609-19,  May  I969. 

[8]  A.  Rosenfeld  and  J.  P.  Strong,  A grammar  for  maps.  Software  Eng., 

Vol.  2,  pp.  227-39,  (op.cit.) 

[9]  A.  Rosenfeld  and  M.  Thruston,  Edge  and  curve  detection  for  visual  scene 
analysis,  IEEE  Transactions  on  Computers,  V.  C-20,  pp.  252-269,  1971. 

[10]  P.  H.  Swain  and  K.  S.  Fu,  Stochastic  programmed  grammars  for  syntactic 
pattern  recognition,  Pattern  Recognition,  Vol.  4,  pp.  83-100,  1972. 

[11]  B.  Upkln  and  A.  Rosenfeld,  Picture  Processing  and  Psychopictorlcs, 
Academic  Press,  1970. 


10 


PICTURE  STATISTICS  AND  TEXTURES 
C.  Myers  and  0.  R.  Mitchell 

One  might  consider  a picture  to  be  divided  Into  three  components: 
the  "low  frequency,"  the  edges,  and  the  textures.  Much  work  has  been  done  on 
the  slowly  changing  properties  of  pictures  and  on  edge  detection,  but  a 
majority  of  the  work  on  texture  analysis  and  classification  has  been  pre- 
dominately problem-oriented.  We  are  Investigating  the  various  properties 
of  each  of  the  many  problem-oriented  classification  methods  to  determine  the 
best  texture  classification  method  for  general  Images  and  also  to  determine 
a useful  model  of  what  texture  Is. 

Many  different  approaches  have  been  taken  to  texture  classification. 

These  techniques  and  some  of  their  advantages  and  disadvantages  are  noted 
below. 

1.  Fourier  spectra:  An  Image  may  be  represented  by  Its  spectral 

frequency  content.  It  has  been  shown  that  different  land 
classifications  such  as  "rural,  woods,  and  urban"  have  different 
spatial  spectral  characteristics  which  could  be  used  to  classify 
them  [1],  The  problems  vs  I th  this  approach  are: 

a.  the  frequency  specr.ra  require  a large  picture  area  to  be 
accurate  and  only  the  gross  properties  are  measured. 

b.  If  the  textural  pattern  is  slightly  Irregular,  the 
pattern  will  get  lost  l.i  the  noise. 

1 

c.  the  positional  or  phase  information  of  the  picture  is  lost. 

2.  Autocorrelation  function:  This  gives  a measure  of  the  picture's 

correlation  with  Itself  (l.e.,  how  closely  two  points  are  related 


to  each  other).  This  function  has  much  the  same  problem  as  the 
Fourier  spectrum  for  Image  classification. 

3.  Gray  level  measures:  The  mean,  variance,  entropy  and  other  such 

measures  may  be  calculated  for  textural  areas  and  the  area  clas- 
sified by  these  measures.  Many  classes  of  texture  measures  for  an 

if 

assortment  of  problems  have  been  tabulated.  Gray  level  measures 

2 3 

have  been  successful  for  textural  classification.  * The  majority 
techniques  still  presuppose  that  the  boundaries  of  the  object  In 
question  are  known.  Also,  statistical  measures  of  this  type  don't 
retain  much  of  the  spatial  relationship  Information  of  the  various 
pixels. 

k.  Joint  probability  density  function  and  related  measures:  Harallck^ 

proposed  that  a joint  density  function  be  produced,  relating  a pixel 
to  another  pixel  In  a given  direction  and  distance  away.  From  the 
density  function  he  calculates  \k  features  for  each  direction  and 
distance  chosen  which  he  then  ’jses  for  classification  of  the  area. 
This  technluqe  has  the  advantage  that  the  features  are  spatially 

dependent  (as  opposed  to  tho^e  derived  from  the  frequency  spectrum) 

/ 

S 

and  that ^a  small  vector  of  numbers  Is  used  to  classify  the  area. 

The  prlmary  disadvantage  Is  that  a large  area  Is  needed  to  calculate 
the  measures  and  the  locality  of  texture  becomes  fuzzy.  Also,  two 
textures  may  occur  In  the  area  of  Interest  and  any  classification 
attempt  would  prove  futile. 

5.  Local  operators  and  templates:  One  way  to  classify  Images  Is  to 
apply  some  local  operator  to  a picture  and  classify  according  to 
the  result  of  the  operator.  Examples  of  local  operators  are  the 
gradient  and  Laplacian:  one  applies  the  operator,  thresholds  the 
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output  and  hopefully  has  the  edges.  Rosenfeld  used  this  technique 
to  find  the  areas  between  objects  of  similar  mean  [7].  His  operator 
was  the  sum  of  the  absolute  value  of  the  difference  between  a point 
and  Its  neighbors.  This  function  was  then  thresholded  (anything 
below  a given  threshold  value  was  considered  a non-edge)  and,  on 
a limited  class  of  pictures,  he  obtained  the  objects. 

The  advantage  of  using  local  operators  is  that  the  textural 
Information  will  remain  localized  so  that  any  further  classification 
will  be  on  a local  basis.  The  disadvantage  of  this  method  is  that 
one  has  to  choose  his  operator,  and  the  choices  are  infinite.  The 
choice  of  the  operator  is  quite  Important,  and  due  to  the  vague 
characteristics  of  texture,  the  choice  is  not  intuitively  obvious. 
Also,  the  choice  of  operators  will  be  extremely  resolution  and 
magnification  dependent. 

Each  of  the  above  techniques  has  Its  advantages  and  disadvantages.  We 
are  trying  to  combine  techniques  in  hopes  of  better  and  more  general  results. 
Some  present  research  Includes: 

1.  Harallck^  chose  fixed  distances  between  points  for  calculation  of 
the  joint  probability  density  function  (spatial  dependence  matrix). 
We  are  deve' oping  adaptive  measures  which  use  the  autocorrelation 
function  results  to  control  the  distance  and  region  for  out  density 
function  and  local  operator  calculations.  This  should  provide  some 
immunity  to  changes  in  resolutions. 

2.  We  have  chosen  a sample  of  sixteen  basic  textures  from  Brodatz^. 

We  plan  to  calculate  basic  gray  level  measures,  Ha ra  1 ick's  measures, 
and  others  as  yet  undefined  measures  and  determine  which  of  these 
measures  are  most  predictable  and  useful.  It  is  hoped  this  will 
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also  lead  to  definitions  of  texture  prlmatlves. 

3*  Rosenfetd  has  shown  that  applying  one  local  operator  to  a picture 
and  thresholding  wl  ’ I be  fairly  successful  for  certain  classes  of 
pictures.  We  are  attempting  to  use  more  than  one  local  operator 
simultaneously  (of  varying  size  and  type)  and  classify  areas 
according  to  the  results  of  these  many  operators  (simitar  to  the 
mul 1 1 spectra  1 classification  problem).  Through  these  techniques, 
we  hope  to  gain  Insight  Into  the  nature  of  texture  prlmatlves  jnd 
to  what  texture  Is. 
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IMAGE  RESTORATION  BY  RECURSIVE  ESTIMATION 
A.  C.  Kak  and  D.  Panda 


The  Problem 

A degraded  Image  can  be  represented  by  the  equation 
y(l,J)  - s(l,j)  X g(!,j)  + n(i,j) 

00  00 

«=  E £ s(l-k,J-ft)  g(k,£)  + n(l,J) 

— 00  —00 

- x(l,j)  + n ( I , j ) , 1<l«n  , 

- original  Image  gray  level 


(1) 


« point  spread  function  of  the  Imaging  medium 

2 

* independent  noise  of  variance  On  and  mean  zero 


where  s (I , j) 

g (I »J) 

n ( I ,j) 

y(I,j)  - distorted  Image 

The  goal  here  Is  to  derive  the  linear  mean  square  recursive  estimate  (LMSE)  of 
the  Image  s (I , j ) In  terms  of  the  observations  y(k,  ),  l<k<m  , l<££n. 

I I . Image  Model 

The  original  Image  Is  assumed  to  be  stationary  and  has  an  exponential 
auto-correlation  function  with  vertical  correlation  coefficient  py  and  hori- 
zontal correlation  coefficient  ph>  It  can  be  shown  that  s(i,J),  LMSE  ofs(i,J) 
in  terms  of  the  signal  s(kTJl)  at  all  points  (M)*  (1  J ) , Is  a 1 1 near  combination 
of  the  8 neighbors  of  s(i,j),  the  coefficient  of  the  vertical,  horizontal , and 
diagonal  neighbors,  0^,0^,  and  arf  respectively,  are: 

P.. 
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and,  s ( I , J)  « s(l,J)  + z(l,J),  where  E[z(l  ,J  )z(k,£)]  = 0 for  (I  ,J)i*(k,fc) . 
This  can  be  expressed  In  vector  form  as 


Aj,  S(0  «=  Av  ?(l+l)  + Ay  1(1-?)  - ?(|) 

where 
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Rearranging  (3)  we  have 


*<■>  “ \]  Ah  !>(l-l)  - S(l-2)  + A^1  if(l-l) 

IN.  Solution  for  A Simple  Degradation 

In  Eq.  (1)  let  us  consider  a special  class  of  point  spread  function 
representing  degradation  due  to  camera  motion,  where 

g(M)  ■ i , k=o  , i _<  i,  <_  o 

0 , ki<0 


then  the  observation  vector  is 


y(l)  - G!>(i ) + fi(|) 
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where  G «* 


0 


g(o,o)  g(o,i)  ....  g(o,tQ) 


g(o,lQ) 


g(o,i) 


g(o,o) 


We  expect  the  recursive  LHSE  of  £(i)  in  terms  of  ^(i)  to  be 

■*  t 1 

S(i)  = Cj  S(i-l)  + c2  S ( i — 2 ) + r Y ( I — 1 ) 


and  hence  the  error  vector  is 


t(i)  - S(i)  - t(i-l)  - f(i-2)  + A~’  ?(i-i) 

t * 

-Cj  S(i-i)  - C2  S ( i —2 ) - r Y(i-i) 


It  can  be  shown  that  Cj  and  C2  can  be  determined  in  terms  of  r(i)  by  using 
orthogonality  principle  and  making  $(i)  - S(i)  orthogonal  to  ?(k)  for  ail 
k?*(i-i ) which  gives 

a . , £ £ 

s(i)  = Ay  (i-r(i)G)"  {(l-r(i)G) AH-r(i)G>  s(i-i)  - s(i-2) 

+ a;1  (i-r(i)G)'1  r(i)  Y(i-i) 

-►  -*■ 

A A -*■ 

Finally  i(i)  can  be  obtained  by  making  S ( i ) - S(i)  orthogonal  tc  Y(i-l), 

IV.  Future  Work 

We  are  working  on  the  following: 

(i)  Formulation  and  solution  of  the  restoration  problem  based  on  3 past 
neighbors  instead  of  8 neighbors. 

(ii)  Application  of  the  method  to  motion  degraded  images  (on  a digital 
computer) . 
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IMAGE  MODELING  BY  INDEPENDENT-INCREMENT  PROCESSES,  II 
T.  S.  Huang  and  J.  Uban 

I.  INTRODUCTION 

As  we  mentioned  in  a previous  report  [I],  one  way  of  viewing  DPCM  is 
that  it  is  the  best  coding  method  for  sources  which  can  be  modeled  by 
I ndependf nt- increment  processes.  Such  a source  is  completely  characterized  by 
rj(A),  the  probability  distribution  of  the  increments  A^'s.  In  that  report 
we  presented  some  measured  p(A)‘s  and  described  several  easily  impiementab!^ 
suboptimum  variable-iength  codes  for  the  A's,  Including  Laemmei  code 
(L^-codes),  Goionb  codes  (G^-codes),  and  Hasier  codes  (H^-codes),  where  N is 
the  basic  block  size.  It  is  the  purpose  of  the  present  paper  to  report  some 
further  experimental  results  on  the  efficiencies  of  these  codes. 


■ 

’ 

I 

:> 

; 


i I . EXPERIMENTAL  RESULTS 

Four  pictures  (part  of  an  ERTS  frame  of  the  Chicago  area-Green  channel, 
a low-aititude  aerial  photo  of  several  ships  In  a harbor,  a portrait,  and 
a crowd  scene)  were  sampled  at  three  different  resolutions:  128  x 128, 

256  x 256,  and  512  x 512  sampler  per  picture.  Each  sample  is  quantized  to 
7 bits.  The  A^'s  ranged  from  -127  to  127.  A set  of  messages  (m^;  k **  i , 

2,...,  255}  were  identified  with  the  A^'s  in  the  following  way: 

A *=  0 -*■  rri| 

A - k -*•  m„. 

A " "k  * m2k+i 

k - 1 ,2 i 27 

1 

.1 

The  messages  were  then  coded  with  various  L^-,  G^-,  and  H^-  codes  in  the 
manner  described  in  Ref.  [1].  The  resulting  bit  rates  are  listed  InTables  1-4. 
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The  lowest  bit  rate  in  each  code  type  is  boxed.  Also  shown  are  the 
entropies.  The  entropies  are  plotted  against  linear  resolution  in  Fig.  i. 

III.  DISCUSSIONS 

The  p^  = Pr(m^)  of  the  ERTS  frame,  the  portrait,  and  the  crowd  are 
roughly  exponential,  while  that  of  the  aerial  phote  contains  a very  large 
Pj  plus  a more  or  less  negative-power  tail.  Therefore,  we  would  expect  that 
L-  and  G-  codes  are  more  efficient  than  H-  codes  for  the  first  three  Images 
and  the  reverse  Is  true  for  the  last.  This  is  Indeediy  largely  borne  out 
in  the  experimental  data  In  Tables  l-^». 

We  may  also  make  the  following  more  specific  observations: 

1)  In  all  cases,  the  best  G-code  Is  more  efficient  than  the  best  L-code. 

2)  While  the  optimum  value  for  N In  the  cases  of  the  L-  and  the  G-  codes 
generally  changes  from  image  to  Image  and  from  resolution  to  resolution, 
that  in  the  case  of  the  H-  codes  is  almost  always  I.  Moreover,  the  bit  rate 
for  the  Hj  - code  is  in  no  cases  more  than  5%  above  that  of  the  optimum  G- 
code.  Therefore,  If  a single  code  Is  to  be  used  for  a variety  of  Images, 
the  Hj  - code  seems  a good  choice. 

3)  When  the  lineai  resolution  of  the  Image  Increases,  the  entropy  per  A 
decreases  roughly  along  a straight  line  (Fig.  I). 

REFERENCE 
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TABLE  2.  Low-Altitude  Aerial  Photo 
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Fig.  1 DPCM  Entropy  as  a Function  of  Resolution 
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SATELLITE  IMAGE  FORMATION  MODEL 


0.  R.  Mitchell  and  P.  L.  Chen 

IMAGF.  FORMATION  MODEL 

The  image  of  the  earth's  surface  (Chicago  area)  shown  in  Fig.  i has  been 
obtained  from  the  ERTS  satellite.  As  we  can  see,  such  images  are  corrupted 
by  atmospheric  effects  and  cloud  formations.  These  "noise"  effects  on  the 
signal  are  not  additive  in  nature  but  represent  muitipi ication  and  convo- 
lution of  noise  and  signal. 

An  approach  to  extracting  signals  from  the  multiplied  and  convolved 
noise  picture  has  been  presented  by  A.  V.  Openhein,  R.  W.  Schafer  and  T.  G. 

Stockham  in  their  paper  "Nonlinear  filtering  of  multiplied  and  convolved 
slgnias."  (Proc.  IEEE  56,  1264-1291,  Aug.  1968). 

We  are  using  this  approach  to  remove  the  effects  of  clouds  and 
atmosphere  in  ERTS  images.  We  actually  have  a three  dimensional  signal 
available  (2  spatial  dimensions  plus  the  four  spectral  bands  represent! ng  the 
third  dimension).  This  opens  the  possibility  of  using  three  dimensional 
filters  to  remove  the  effects  of  noise  from  the  signal. 

•The  filter  design  requires  that  we  know  the  statistics  of  the  noise  and 
the  signal.  If  the  noise  is  produced  by  cloud  cover,  the  information  is  most 
likely  avaiiabie  in  the  spectral  signature;  if  the  noise  is  produced  by 
atmospheric  turbulence,  these  statistics  have  already  been  studied  and 
documented.  (B.  L.  hcGlamery,  Restoration  of  turbulence-degraded  'mages, 

J.  Opt.  Soc.  Amer.  57*  293-297»  March  1967)* 

Prior  to  working  with  the  ERTS  Images  two  types  of  simulated  noisy 
pictures  are  used  as  our  test  samples. 

! I 
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(1)  Real  physical  stimulation: 


Images  containg  multiplicative  noise  are  obtained  by  taking  photo- 
graphs through  either  homogeneous  or  non-homogeneous  srroke  as  shown  in 
Fig.  2. 

The  power  spectrum  of  the  smoke  and  the  power  spectrum  of  the  signal 
are  then  obtained  and  an  optimum  Weiner  filter  designed  for  the  homomorphic 
filtering  operation  to  separate  the  smoke  from  the  signal. 

(2)  Computer  simulation: 

A two  dimensional  pattern  of  random  numbers  Is  generated  and  filtered 
to  given  tnem  the  same  power  spectrum  as  typical  smoke,  haze  or  clouds. 

These  correlated  random  numbers  are  then  used  to  distort  digitized 
Images.  The  dependence  of  various  filtering  operations  and  using  diffe  ent 
type  noises  can  then  be  studied. 

It  is  hoped  that  these  studies  in  filtering  and  image  and  noise 
statistics  will  enhance  our  insight  Into  image  models. 
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Fig.  1 ERTS  Picture  with  Cloud  Cover. 


> 


Fig.  2 Laboratory  Generated  Image  with  Smoke 
(multiplicative  noise). 
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ITERATIVE  IMAGE  RESTORATION,  II 
T.  S.  Huang  and  S.  P.  Berger 


I.  INTRODUCTION 

Although  much  of  the  work  in  image  restoration  has  been  done  In  the 
area  of  linear  spatial ly-invariant  Inverse  filters.  It  Is  evident  from  past 
work  [1]  that  In  order  to  restore  severly  degraded  images  due  to  camera 
motion,  lens  aberration,  atmospheric  turbulence,  etc.,  we  have  to  have 
recourse  to  linear  spatially-varying  and  nonlinear  Inverse  filters  and  to 
make  use  as  much  as  we  can  of  any  a priori  knowledge  we  may  have  about  the 
original  image. 

The  implementation  of  linear  spatially-varying  and  nonlinear  Inverse 
filters  are  generally  very  time  consuming.  However,  the  use  of  Iterative 
methods  may  provide  a relatively  efficient  solution.  In  a previous  report 
[2]  we  have  presented  an  example  of  restoring  linearly  degraded  Image  by  a 
particularly  attractive  iterative  technique,  the  projection  method.  It  Is 
the  purpose  of  the  present  paper  to  report  some  further  examples.  Experi- 
mental results  Indicated  that  this  method  could  potentially  achieve  super 
resolution  In  the  presence  of  noise. 

II.  EXPERIMENTAL  RESULTS 

In  this  section  we  present  two  examples  of  the  use  of  the  projection 
method.  In  all  cases,  the  only  a priori  information  we  made  use  of  was  the 
fact  that  fj  _>  0. 

Our  first  example  is  the  one-dimensional  signal  shown  in  Fig.  I (a). 

For  the  purpose  of  digital  processing,  32  samples  were  taken  from  It: 
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V f3|*  We  note  that  the  signal  consists  of  t*o  pulses  of  ampli- 

tude 7 separated  by  a gap  of  three  samples  of  amplitude  0.  This  signal 
was  smeared  over  10  samples,  resulting  In  the  degraded  signal  g?  (thesolld 
curve  In  Fig.  1(b)).  Specifically, 


The  projection  method  was  applied  to  this  degraded  Image  with  an  Initial 
guess  ?(0)  - 0.  The  results  after  I and  10  cycles  of  Iterations  are  shown 
In  Fig.  1(b).  Note  that  the  pulses  were  well  resolved.  Next,  noise  was 
added  to  the  smeared  signal  (the  solid  curve  of  Fig.  1(b))  to  get  a noisy 
smeared  signal  (the  solid  curve  of  Fig.  2).  The  noise  was  Independent  of 
the  signal,  white,  and  has  a uniform  distribution  with  zero  mean  and  a 
standard  deviation  an  = .5.  The  peak-signal -to- rms-no I se  ratio  was  then 
about  23  db.  The  projection  method  was  applied  to  this  noisy  smeared  signal 
with  an  initial  guess  ?(0)  - 0.  The  result  after  10  cycles  of  iterations 
was  shown  In  Fig.  2.  Note  that  the  two  original  pulses  were  resolved. 
However,  several  spurious  pulses  with  smaller  amplitudes  appeared  In  the 
restoration.  We  note  In  passing  that  although  the  frequency  response  of  the 
smearing  Is  not  strictly  bandllmited,  the  presence  of  the  noise  makes  It 
pratlcal ly  so.  Therefore,  the  result  In  Fig.  2 Indicates  that  the  projection 
method  could  potentially  achieve  super  resolution. 

Our  second  example  Is  the  Image  of  a cross  shown  In  Fig.  3(a).  The 
digitized  Image  contains  128  x I2C  samples.  The  background  Is  at  gray  level 
255  and  the  center  of  the  cross  at  gray  level  0.  The  halftone  displays  In 
Figs.  3 and  k were  obtained  on  a Gould  electrostatic  printer.  This  original 
Image  was  linearly  smeared  In  the  horizontal  direction.  The  smearing  was 
spatial ly- In variant  along  any  particular  row.  It  was,  however,  different 
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from  row  to  row.  At  the  center  of  the  Image,  there  was  no  smearing.  The 
duration  of  the  smearing  Increased  linearly  as  we  moved  away  from  the 
center.  Noise  was  then  added  to  the  smeared  Image  to  obtain  the  noisy 
smeared  Images  of  Fig.  3(b)  and  4(a).  The  noise  was  Independent  of  the 
signal,  white,  and  Gausslanly  distributed  with  zero  mean.  The  peak-slgnal- 
to-rms-noise  ratios  for  the  Images  In  Fig.  3(b)  and  4(a)  are  36  db  and  30db, 
respectively.  The  projection  method  was  applied  to  these  no*  , '•neared 
Images,  with  the  Initial  guess  equal  to  the  degraded  Image.  The  results 
after  I,  5,  and  15  Iterations  are  shown  In  Figs.  3(c)-(e)  and  4(b)-(d). 

We  note  that  in  both  cases  the  Image  was  sharpened  up  after  a sufficient 
number  of  cycles  of  Iterations.  However,  the  noise  was  amplified.  Generally, 
as  we  do  more  and  more  Iterations,  the  Image  becomes  sharper  and  sharper, 
but  In  the  meantime  the  noise  becomes  more  and  more  amplified.  One  way  to 
decide  on  the  optimum  number  of  Iterations  Is  simply  to  look  at  the  result 
after  each  cycle  of  iteration,  and  choose  the  visually  best  one.  This  pro- 
cedure will  be  greatly  facilitated  If  we  are  working  on  a man-machine 
Interactive  system  with  a fast  display  unit. 

III.  CONCLUDING  REMARKS 

We  have  presented  some  further  results  of  applying  the  projection 
method  to  Image  restoration.  To  reiterate,  this  Iterative  method  has  many 
advartages,  specifically: 

1.  It  always  converges.  When  the  solution  Is  unique,  It  will  converge  to 
It.  When  the  solution  Is  not  unique,  It  will  converge  to  that  solution 
which  Is  closest  to  the  Initial  guess  solution. 

2.  It  can  readily  take  advantage  ot  a priori  Information  about  the  image- 

3.  The  computation  time  requirement  Is  not  excessive. 
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Our  experimental  results  Indicated  that  the  projection  method  could 
potentially  achieve  super  resolution. 
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FIGURE  CAPTIONS 

Fig.  1 (a)  original  signal 

(b)  smeared  and  restored  signals. 

Fig.  2 Noisy  smeared  signal  and  its  restoration 

Fig.  3(a)  original  Image 

(b)  noisy  smeared  Image  (peak-si gnal-to-rms-noise  ratio  = 36  db) . 

(c)  restored  Image  (I  cycle  of  Iterations) 

(d)  restored  image  (5  cycles  of  Iterations) 

(e)  restored  Image  (15  cycles  of  iterations). 

Fig.  4(a)  noisy  smeared  Image  (peak-signal-to-rms-noise  ratio  = 30  db) 

(b)  restored  Image  (1  cycle  of  iterations) 

(c)  restored  image  (5  cycles  of  Iterations) 

(d)  restored  Image  (15  cycles  of  Iterations). 
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Fig.  1(b)  Smeared  and  Restored  Siqna’.s. 
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Fig.  2 Noisy  Smeared  Signal  and  its  Restoration. 
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(c)  restored  image  (1  cycle  of  iterations) 

(d)  restored  image  (5  cycles  of  iterations) 

(e)  restored  image  (15  cycles  of  iterations). 
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noisy  smeared  image  (peak-signal-to-rms-noise  ratio  = 30  db) 
restored  image  (’  cycle  of  iterations) 
restored  image  (5  cycles  of  iterations) 
restored  image  (15  cycles  of  iterations). 


A BRANCH  AND  BOUND  CLUSTERING  ALGORITHM* 
K.  Fukunaga  and  P.  M.  Narendra 


I.  INTRODUCTION 

Clustering  may  be  viewed  as  a combinatorial  optimization  problem  In 
that,  of  the  MN/Mi  ways  In  which  N objects  ma>  be  assigned  to  M classes,  the 
"best"  classification,  (I.e.,  one  thct  extremlzes  the  value  of  a given 
criterion)  Is  to  be  determined.  One  way  to  find  a 'good'  classification  is  to 
adopt  hill  climbing  techniques  which  Iteratively  Improve  the  solution  to 
optimize  the  criterion.  Most  clustering  algorithms  described  in  the  literature 
[Mi  [2]  belong  to  this  class.  While  these  algorithms  are  computationally 
efficient,  they  do  not  guarantee  a globally  optimum  solution. 

Exhaustive  enumeration  of  ail  solutions,  can,  of  course  determine  the 
globally  optimum  classification.  But  the  computation  required  Is  prohibitive. 
The  method  of  branch  and  bound  [4]  - [7]  Is  a much  more  reasonable  approach  to 
combinatorial  optimization  problems.  In  what  follows  we  present  a summary  of 
the  development  of  a practical  branch  and  bound  clustering  algorithm,  which 
finds  the  globally  optimum  classification  while  remaining  computationally 
feasible. 

1 1.  THE  BRANCH  AND  BOUND  ALGORITHM 

*-et  Xjj  be  N vectors  of  dimension  n.  These  vectors  ure  to  be 

clustered  Into  M classes  denoted  1,  2 K.  Let  ujj  be  the  class  to  which  Xj 

Is  assigned  by  the  clustering,  I.e.,  u,  e {1,2 M}.  Let  the  classification 

ft  be  defined  by  T 

ft  * [w.  W).  ...  10J 

* 1 2 N 

This  work  was  supported  In  part  by  the  National  Science  Foundation  under 

Grant  GJ-35722. 
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x. ■ [xj  xj  ...  xJ]T. 


The  clustering  criterion  is  a scalar  function  J(ft;X)  of  Q and  X.  The  cluster- 
ing problem  then  is:  for  a given  configuration  X,  determine  the  classification 


ft  , such  that 


J (ft* ;X)  * Min  J (Si; x) 
ft 


To  seek  the  above  minimum  by  exhaustive  enumeration  requires  the  eval- 
uation of  the  criterion  for  MN/M!  classification  Si.  However,  we  may  llmltthe 
numbers  of  classification  to  be  enumerated  as  follows: 


Define  a partial  criterion  as 


J (1  ,k)  - J(u>1,<.J2»*.*»(V»  * 


a function  of  the  first  k vectors  and  their  classification.  Further,  suppose 
that  for  a given  (u^  , . . . » 


J (1 ,1 ) < J(l,2)  < ...  1 JO.N)  . 


Let  B be  an  upper 


bound  on  the  optimum  value  of  J(fl;X),  l.e., 


B > J (ft  ;X) 


Now,  if,  for  a given  partial  classification:  [u>® , to® wkJ  ’ (k  i N)  * 


we  have, 


j (tOj , ....  w®;  x.)  > B» 


then  all  classifications  of  the  form  [w®,  •••»  wk*  Wk+1 ®re  SUb 


optimal.  This  Is  because, 


J(to® wk+1,  ....  Wj,)  > J(M*...«k)  > B , 


j 


from  (3).  Hence,  if  such  an  Inequality  as  (5)  is  satisfied  at  a certain 
stage  k,  M possible  trial  classifications  are  found  to  be  suboptimal, 
and  do  not  have  to  be  enumerated.  So  the  clustering  problem  reduces  to  that 
of  finding  an  efficient  way  of  computing  the  upper  bounds  B and  enumerating 

the  partial  sequences  taj,  so  that  the  inequality  in  (5)  is 

satisfied  for  a large  number  of  the  enumerated  sequences. 

The  algorithm  is  an  alternating  sequence  of  partial  classification 
generation  and  partial  classification  evaluation  steps.  All  possible 
classifications  are  either  explicitly  evaluated  or  implicitly  rejected. 

The  optimum  classification  is  thereby  determined.  A detailed  account  of 

the  algorithm  implemented  for  the  withln-class  scatter  criterion  may  be 
found  in  [8]. 

III.  EXPERIMENTAL  RESULTS 

The  clustering  algorithm  was  applied  to  120  samples  from  a bivariate 
distribution.  Figure  1 shows  the  optimum  classification  generated  when  the 
algorithm  was  applied  to  the  data  to  cluster  It  Into  two  clusters.  Exhaustive 
enumeration  in  this  case  would  have  required  the  evaluation  of  2120  different 
classification.  The  clustering  algorithm  obtai ned  the  optimal  classification 
in  27  seconds  of  CPU  time  on  the  CDC  6500.  This  is  ample  indication  that  the 
algorithm  Is  very  viable  even  for  large  sample  sizes. 

iV.  CONCLUSION 

The  theoretical  part  of  this  project  has  been  completed.  A detailed 
report  will  be  published  [8],  Further  work  is  underway  to  apply  the  branch 
and  bound  optimization  techniques  to  several  problems  In  Pattern  Recognition. 
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APPLICATION  OF  SINGULAR  VALUE  DECOMPOSITION* 
TO  PATTERN  RECOGNITION 


K.  Fukunaga  and  G.  V.  Sherman 

I . INTRODUCTION 

Classification  of  sampled  Images  by  decision-theoretic  methods  Is 
usually  preceded  by  a feature  extraction  technique  for  reducing  the 
dimensionality  of  the  problem. 

Unfortunately  most  existing  statistical  methods  of  feature  extraction 
require  the  calculation  of  eigenvectors  of  the  same  dimensionality  as  the 
original  data.  Specific  examples  are  factor  analysis  [1],  mixture  normal- 
ization [2],  and  the  subspace  method  [33 . In  the  case  of  a small  16  x 16 
subpicture,  the  above  mentioned  eigenvectors  would  be  256-dimenslonal ; and 
computation  time  would  be  prohibitive. 

In  this  report,  we  propose  to  operate  on  the  rows  and  columns  of  the 
picture  Independently,  so  that  the  mathematical  manipulations  would  take 
place  In  a vector  space  of  dimensionality  equal  to  that  of  the  rows  or 
columns  of  the  picture. 


M. 

AM 


DESCRIPTION  OF  THE  METHOD 


Given  the  classes  w^  (k*I 


from  the  training  set  R 


(k) 


AcR 


(k) 


(Ajw  } 


...,M),  we  form  M standard  picture  matrices 
of  each  class  according  to 

0) 


where  E denotes  expectation  and  A denotes  randan  pictures.  All  picture 

matrices  will  be  assuneo  to  be  n x n for  simplicity.  Next  n x n matrices 
* 
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$(k),  t and  A^  (the  last  being  diagonal)  are  calculated  by 
A(k)fl(k)T  {(k)  . ,:k)(A(k),J 


A(k)  „+((A<k>)2]l/2 


¥(k)  = A 


(k)T^)(A(k>)-1 


tor  k = 1,...,M.  Let  A.  be  the  random  picture  matrix  which  is  to  be  clas- 
(k) 

s If  led.  K.  Is  calculated  for  k = l,...,n  by 

r,  A (5) 

A Is  classified  into  the  class  w.  for  which  classification  criterion 

— k 

* , . 

J (£)  is  minimized. 

J (k)  = min  Z Z (k,  . ^ - X.  . ^ )2  -*■  A e w (6) 

£ i-l  j=l  ,J  IJ  k 

(£)  00  00 
where  k. . is  the  ljth  element  of  K and  X.  , Is  the  ijth  element  of 
-ij  “ U 

(£) 

Av  . m2  is  the  number  of  features  used. 

For  m = n the  classification  criterion  becomes 

J*(k)  « min  tr  [(A  - AU))  (A  - AU))T]  - A wfc  (7) 

2 

which  is  simply  the  nearest-mean  criterion  in  an  n dimensional  vector  space. 
The  success  of  this  method  will  be  demons  fated  If  a significant  reduction 
In  the  number  of  features  can  be  achieved  without  an  excessive  Increase  In 
classification  error. 


i I i.  EXPERIMENTAL  RESULTS 

The  numerals  of  the  Munson  multi-author  alphanumeric  data  were  used. 

The  second  and  third  sample  of  each  numeral  submitted  by  each  author  con- 
stituted the  training  set  of  98  sampler  per  class.  The  first  sample  of  each 


r 
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numeral  submitted  by  each  author  made  up  the  test  set  of  49  samper  per 
class.  Of  course  there  are  10  classes  corresponding  to  the  10  numerals. 

Each  numeral  contains  24  x 24  points. 

in  order  to  investigate  the  feature  extraction  capabilities  of  this 
technique,  the  experiment  was  repeated  for  8 values  of  m.  Figure  1 is  a 
plot  of  classification  accuracy  versus  number  of  features.  Although  at 
first  glance  79%  classification  accuracy  seems  unimpressive,  to  the  author's 
knowledge  no  other  group  has  achieved  significantly  better  results  with  this 
data  base  using  such  a general  approach  [4].  Granted  ad  hoc  methods  do  give 
somewhat  better  accuracy,  our  purpose  is  to  demonstrate  a general  method 
applicable  to  a wide  variety  of  data.  Moreover  Hussain,  et  al . have  shown 
that  classification  errors  depend  very  much  on  how  the  training  set  is 
selected  from  this  particular  data  base  which  further  complicates  comparisons 
[4].  Furthermore  no  effort  was  made  to  normalize  the  size,  line  width,  or 
rotation  of  the  Munson  numerals.  Fig.  1 indicates  that  a reduction  from 
576  to  144  features,  a 75%  reduction,  causes  only  a k%  reduction  in  clas- 
sification accuracy.  Thus  the  method  can  Indeed  be  considered  successful. 

IV.  AREAS  FOR  FURTHER  RESEARCH 

Instead  of  using  the  SVD,  one  could  expand  a picture  into  the  principle 
components  of  both  rows  and  columns  simultaneously.  That  is  $ and  y would 
now  be  the  eigenvector  matrices  of  the  autocorrelation  matrices  of  the 
columns  and  rows  of  the  A respectively.  Hopefully  a two-dimensional 
extension  of  the  mixture  normalization  or  subspace  method  could  be 
developed.  Furthermore,  the  second  order  statistics  of  the  A could  be  used 
to  predict  which  of  the  terms  In  (6)  are  small  and  thus  negligible. 

These  approaches  are  currently  being  pursued. 
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FACILITIES 


The  Purdue  IMP  arrived  and  was  connected  to  the  net  during  this  re- 
search period  but  interfacing  out  computer  to  the  IMP  was  not  completed. 

We  are  currently  debugging  the  hardware  interface  between  our  PDP  11A5  and 
the  IMP.  All  of  our  computing  over  the  net  was  done  through  a leased  1200 
bit/sec  telephone  line  connected  to  the  Illinois  ANTS  system.  Some  high 
volume  processing  work  was  done  locally  due  to  Lhe  low  transmission  rate  of 
the  leased  line. 

We  expect  to  complete  our  IMP  interface  debugging  in  the  immediate 
future.  V/e  are  simultaneously  developing  the  use  of  both  ELF  and  ANTS  opera- 
ting systems  on  our  machines.  Present  facilities  available  to  us  include: 

1  PDP  11M5  computer;  32K  core;  disk  operating  system  software 

1 20-million  word  disk 

2 9-track  tape  drives;  800  and  1 600  bp i densities 

1 7-track  tape  drive;  556  and  800  bpi  densities 

3 CRT  terminals 

1 card  reader 

1 paper  tape  reader 

1 line  printer;  600  lines  per  minute,  impact  type 

1 electrostatic  graphics  hard  copy  system,  200  points  per  inch 

\ 

1 video  display  with  disk  storage;  3 color  refreshed  display,  512 x 512 
points,  8 bits/color/point;  graphics  overlay  and  interact  1 ve  capabilities 

1 flying  spot  scanner 

A few  hardware  interfaces  between  the  PDP  11A5  and  the  peripherals  are 
yet  to  be  completed.  We  predict  that  basic  network  data  transfer  capabi  1 ities 
will  be  available  in  early  - 1975  and  complete  hardware  and  software 


capabilities  will  be  available  in  mid  - 1975. 
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SOFTWARE  DEVELOPMENT  FOR  IMAGE  PROCESSING 


We  are  developing  an  Integrated  collection  of  picture  processing  pro- 
grams with  a sensible  mix  of  flexibility  and  standardization  for  processing 
over  the  ARPA  net.  The  file  structure  chosen  for  large  Images  Is  compatible 
with  that  used  by  the  Laboratory  for  the  Application  of  Remote  Sensing  (LARS) 
This  allows  us  to  obtain  ERTS  data  directly  from  LARS  and  to  use  the  clas- 
sification programs  already  developed  by  LARS. 

Features  presently  incorporated  In  our  software  Include: 

1.  Fast  Fourier  Transform  (128  x 128)  maximum  picture  size) 

2.  Autocorrelation 

3.  Histogram,  joint  probability  density  function 

4.  Convolution 

5.  Template  matching 

6.  Utl 1 i ty  programs 

7.  Graphic  output 

a.  Three-dimensional  plots 

b.  Contour  plots 

c.  Grey-level  picture  on  electrostatic  printer 
Work  In  progress  and  future  plans  include: 

8.  FFT  for  512  x 512  pictures 

9.  Two-dimensional  filtering 

10.  Picture  Editing 

11.  Histogram  Equl 1 Izatlon 

12.  Picture  creation  and  manipulation  by  use  of  very  simple  commands 

13.  Clustering  algorithms 

14.  BLOB  algorithms  for  Image  segmentation. 
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Applications  In  Pattern  Recognition,"  to  appear  In  IEEE  Trans,  on 
Information  Theory,  (with  L.  D.  Hostetler). 
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Theory,  (with  D.  L.  Kessell). 
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Academic  Press,  Spring  1975. 
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di 


MISSION 

of 

Rome  Air  Development  Center 


RADC  is  the  principal  AFSC  organization  charged  with 
planning  and  executing  the  USAF  exploratory  and  advanced 
development  programs  for  information  sciences,  intelli- 
gence, command,  control  and  communications  technology, 
products  and  services  oriented  to  the  needs  of  the  USAF. 
Primary  RADC  mission  areas  are  communications,  electro- 
magnetic guidance  and  control,  surveillance  of  ground 
and  aerospace  objects,  intelligence  data  collection  and 
handling,  information  system  technology,  and  electronic 
reliability,  maintainability  and  compatibility.  RADC 
has  mission  responsibility  as  assigned  by  AFSC  for  de- 
monstration and  acquisition  of  selected  subsystems  and 
systems  in  the  intelligence,  mapping,  charting,  command, 
control  and  communications  areas. 


